1. Field of the Invention
The present invention relates generally to an image processing apparatus, and more particularly to a method and apparatus for identifying and localizing an area in relative movement in a scene and determining the speed and oriented direction of the area in real time.
2. Description of the Related Art
The human or animal eye is the best known system for identifying and localizing an object in relative movement, and for determining its speed and direction of movement. Various efforts have been made to mimic the function of the eye. One type of device for this purpose is referred to as an artificial retina, which is shown, for example, in Giocomo Indiveri et. al, Proceedings of MicroNeuro, 1996, pp. 15-22 (analog artificial retina), and Pierre-Francois Ruedii, Proceedings of MicroNeuro, 1996, pp. 23-29, (digital artificial retina which identifies the edges of an object). However, very fast and high capacity memories are required for these devices to operate in real time, and only limited information is obtained about the moving areas or objects observed Other examples of artificial retinas and similar devices are shown in U.S. Pat. Nos. 5,694,495 and 5,712,729.
Another proposed method for detecting objects in an image is to store a frame from a video camera or other observation sensor in a first two-dimensional memory. The frame is composed of a sequence of pixels representative of the scene observed by the camera at time to. The video signal for the next frame, which represents the scene at time t0 is stored in a second two-dimensional memory. If an object has moved between times to and t0 the distance d by which the object, as represented by its pixels, has moved in the scene between t, and to is determined. The displacement speed is then equal to d/T, where T=t1−t0. This type of system requires a very large memory capacity if it is used to obtain precise speed and oriented direction. Information for the movement of the object. There is also a delay in obtaining the speed and displacement direction information corresponding to t1+R, where R is the time necessary for the calculations for the period t0-t1 system. These two disadvantages limit applications of this type of system.
Another type of prior image processing system is shown in French Patent No. 2,611,063, of which the inventor hereof is also an inventor. This patent relates to a method and apparatus for real time processing of a sequenced data flow from the output of a camera in order to perform data compression. A histogram of signal levels from the camera is formed using a first sequence classification law. A representative Gaussian function associated with the histogram is stored, and the maximum and minimum levels are extracted. The signal levels of the next sequence are compared with the signal levels for the first sequence using a fixed time constant identical for each pixel. A binary classification signal is generated that characterizes the next sequence with reference to the classification law An auxiliary signal is generated from the binary signal that is representative of the duration and position of a range of significant values. Finally, the auxiliary signal is used to generate a signal localizing the range with the longest duration, called the dominant range. These operations are repeated for subsequent sequences of the sequenced signal.
This prior process enables data compression, keeping only interesting parameters in the processed flow of sequenced data. In particular, the process is capable of processing a digital video signal in order to extract and localize at least one characteristic of at least one area in the image. It is thus possible to classify, for example, brightness and/or chrominance levels of the signal and to characterize and localize an object in the image.
Another system is also known from WO 98/05002, of which the inventor hereof is also an inventor. This system enables real time detection, location and determination of the speed and direction of movement of an area of relative movement in a scene. It includes a time processing unit of a spatial processing unit in order to determine said speed and direction of movement.
U.S. Pat. No. 5,488,430 detects and estimates a displacement by separately determining horizontal and vertical changes of the observed area. Difference signals are used to detect movements from right to left or from left to right, or from top to bottom or bottom to top, in the horizontal and vertical directions respectively. This is accomplished by carrying out an EXCLUSIVE OR function on horizontal/vertical difference signals and on frame difference signals, and by using a ratio of the sums of the horizontal/vertical signals and the sums of frame difference signals with respect to a K×3 window. Calculated values of the image along orthogonal horizontal and vertical directions are used with an identical repetitive difference K in the orthogonal directions, this difference K being defined as a function of the displacement speeds that are to be determined. The device determines the direction of movement along each of the two orthogonal directions by applying a set of calculation operations to the difference signals, which requires very complex computations. Additional complex computations are also necessary to obtain the speed and oriented direction of displacement (extraction of a square root to obtain the amplitude of the speed, and calculation of the arctan function to obtain the oriented direction), starting from projections on the horizontal and vertical axes. This device also does not smooth the pixel values using a time constant, especially a time constant that is variable for each pixel, in order to compensate for excessively fast variations in the pixel values.
Finally, Alberto Tomita Sales Representative. and Rokuva Ishii, “Hand Shape Extraction from a Sequence of Digitized Gray-Scale Images,” Institute of Electrical and Electronics Engineers, Vol. 3, 1994, pp. 1925-1930, detects movement by subtracting between successive images, and forming histograms based upon the shape of a human hand in order to extract the shape of a human hand in a digitized scene. The histogram analysis is based upon a gray scale inherent to the human hand. It does not include any means of forming histograms in the plane coordinates. The sole purpose of the method is to detect the displacement of a human hand, for example, in order to replace the normal computer mouse by a hand, the movements of which are identified to control a computer.
It would be desirable to have an image processing system which has a relatively simple structure and requires a relatively small memory capacity, and by which information on the movement of objects within an image can be obtained in real-time. It would also be desirable to have a method and apparatus for detecting movements that are not limited to the hand, but to any object (in the widest sense of the term) in a scene, and which does not use histograms based on the gray values of a hand, but rather the histograms of different variables representative of the displacement and histograms of plane coordinates. Such a system would be applicable to many types of applications requiring the detection of moving and non-moving objects.